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Abstract. In this paper, we determine the maximally stable, rotation- 
ally invariant domains on the catenoids C a (minimal surfaces invariant 
by rotations) in the Heisenberg group with a left-invariant metric. We 
show that these catenoids have Morse index at least 3 and we bound the 
index from above in terms of the parameter a. We also show that the 
index of C a tends to infinity with a. Finally, we study the rotationally 
symmetric stable domains on the higher dimensional catenoids. 
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1. Introduction 

Minimal surfaces in the Heisenberg group equiped with a left-invariant met- 
ric have been studied by several authors, see El HJ E] and the references 
therein. Catenoids in the Heisenberg group Nil(3) are complete minimal 
surfaces which are invariant under a one-parameter subgroup of rotations 
with axis the center of the group. They come in a one-parameter family 
{C a ,a > 0} of complete minimal surfaces and were first described in [7] 
and [S] where the authors provide the classification of constant mean curva- 
ture surfaces in the Heisenberg group, invariant under certain subgroups of 
isometries. 

In this paper, we study the stability properties of the catenoids {C a , a > 0}. 
More precisely, we determine the rotationally invariant stable domains of 
the catenoids in Nil(2n + 1), n > 1, with a different behaviour (Lindeloef's 
property) when n = 1 and when n > 2. We also study the Morse index 
of the catenoids in Nil(3). As in [3], the proofs rely in part on a detailed 
analysis of the Jacobi fields induced from the Killing fields of the ambient 
Heisenberg space and from the variation of the parameter a. 

The paper is organized as follows. In Section [21 we give some preliminary 
results. We first recall the basic geometry of the Heisenberg group Nil(3) 
equiped with a left-invariant metric g (see [H] for more details). In order to 
keep our paper self-contained, we derive the differential equation satisfied by 
the generating curves of the catenoids, using a flux formula. In Section El we 
describe the stable rotationally invariant domains on {C a } (Theorem 13. 
The proof uses Jacobi fields. We also give some information on the Gauss 
map of the catenoids {C a }. In Section HI Theorem \4A\ we prove that the 
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catenoids C a ,a > have Morse index at least 3. We bound the index from 
above in terms of a, and we also show that its goes to infinity with a. The 
proof uses Jacobi fields, Fourier analysis and an adapted perturbation of the 
original parametrization of the catenoids. Finally, in Section [SJ we study the 
maximally stable, rotationally invariant domains on the higher dimensional 
catenoids (Theorem 15, ip . 

In the sequel our functions will often depend on the parameter a. We will 
occasionally omit a to keep the notations simpler. In this paper, we only 
consider left-invariant Riemannian metrics on the Heisenberg groups. 

The first author was partially supported by the cooperation programme 
Math-AmSud. The second author would like to thank Institut Fourier 
(Grenoble) for their hospitality during the preparation of this paper. He 
gratefully acknowledges CAPES and CNPq for their financial support. 



2. Preliminaries 

2.1. The 3-dimensional Heisenberg manifold. Let Nil(3) denote the 3- 
dimensional Heisenberg group. This is a two-step nilpotent Lie group which 
can be seen as the subgroup of 3 x 3 matrices given by 
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Using the exponential map, exp : £(Nil(3)) — > Nil(3), and the Campbell- 
Hausdorff formula, 

exp (A) exp (B) = exp (A + B + -[A, B]), Wl, B G £(Nil(3)), 

we can view Nil(3) as M 3 equiped with the group structure * given by 

(x, y, z) * (x', y', z') = (x + x',y + y' , z + z' + -(xy' - x'y)^ , 

with neutral element = (0, 0, 0) and inverse p of p = (a, b, c) given by 
p = (—a, —b, — c). The left-multiplication by p in Nil(3), L p : q i— >■ p ★ q, has 
tangent map 

/ 1 ON 

(1) T q L p = 10 

in the canonical coordinates {x, y, z] of ]R 3 (they are often referred to as 
exponential coordinates). Let {d x , d y , d z } denote the canonical vector fields 
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in R 3 . It follows from the expression ([I]) that the vector fields 

!X{x,y,z) = T L {X)yyZ) (d x ) = d x -\d z , 
Y(x,y,z) = T Q L( Xi y iZ) (dy) = d y + %d z , 
Z(x,y,z) = T L( X:ytZ) (d z ) = d z , 
form a basis of left-invariant vector fields in Nil(3). 

The metric g on Nil(3). From now on, we fix the left-invariant metric g 
on Nil(3) to be such that the family {X, Y, Z} is an orthonormal frame. In 
the coordinates {x,y,z}, this metric is given by 

g = dx 2 + dy 2 + (dz + - (ydx — xdy)) 2 . 

The following properties are well-known and can be found for example in [8] , 
Section 1. Equiped with the left-invariant metric g, the Heisenberg group 
Nil (3) is a homogeneous Riemannian manifold whose group of isometries 
has dimension 4. A basis of Killing vector fields on (Nil(3),g) is given by 

' e = x + yz, 

7] = Y — xZ, 
' < = Z, 

, p = yX - xY + \(x 2 + y 2 )Z. 

The first three vector fields £, r\ and £ correspond to the one-parameter sub- 
groups of isometries generated by right-invariant vector fields in Nil(3), while 
the vector field p corresponds to the one-parameter subgroup of isometries 
defined by 

(3) ipe((x, y, z)) = (x cos9 — ysm9, x sin 6 + y cos 6,z),(x,y,z) £ R 3 ,6 £ R, 

in the representation (R 3 ,*) of Nil(3). We call them rotations around the 
z-axis. Notice that the z-axis is precisely the center of Nil(3). 

2.2. Surfaces of revolution in Nil(3). We say that a surface M in Nil(3) 
is a surface of revolution if M is invariant under the action of the one- 
parameter subgroup {i)e,8 G R} given by ©. We will consider surfaces of 
revolution whose generating curves are graphs t — > (f(t), t) above the z-axis 
in the 2-plane {x,z}, where / is a positive function, and where t varies in 
some interval JcR. They are given by a map 

(4) T(t,0) = (/(*) cos 9J(t) sin 6,t), 
for t € I C R and 9 G [0,2vr]. 

Catenoids, i.e. minimal surfaces of revolution, in Nil(3) are described in 
[5] , using the methods of equivariant differential geometry. They come in 
a one-parameter family of complete minimal surfaces, {C a , a > 0}. For the 
sake of completeness and for later purposes, we will derive the differential 
equations satisfied by the generating curve of a catenoid using a flux formula 
which we now state. 

Proposition 2.1. Let (M n ,g) ^ (M n+1 ,g) be an isometric immersion 
with Riemannian measure p g and normalized mean curvature vector H . Let 
be a relatively compact smooth domain in M . Let i>i n t denote the unit 
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normal to <9f2 in M , pointing inwards, and a g the Riemannian measure on 
dQ induced by g. Then, for any Killing vector field K, on M n+1 , we have 

/ g{K,,v int )da g = -n \ g(JC, H) d/j, g . 
JdQ Jn 

Proof. Let k be the restriction to M of the 1-form dual to /C, i.e. k = 
g(IC, .)\m- Recall that /C is a Killing field if and only if, for any vector field 
X on M, g(D x JC,X) = (0, Proposition 3.2, p. 237). A straightforward 
computation shows that the divergence 8 g n of the 1-form k, for the induced 
metric g on M, is given by 

5 g K = —ng(fC, H). 

The proposition follows from the divergence theorem. □ 

Let M = T(I x [0, 27r]) be a minimal surface of revolution in Nil(3), given 
by an immersion J-(t, 6) as in @, with t E / C M, 9 E [0, 27r]. We can make 
a coherent choice of a unit vector field v tangent to M and orthogonal to 
the circles Ct = J~({t} x [0, 2ir]) in such a way that Proposition 12.11 gives 

(5) / gQC, v) da Ct = / g(1C, v) da CtQ , 
J Ct ■'Cto 

for all t £ I an d for any Killing vector field K, in Nil(3). 

Proposition 2.2. The generating curve of a minimal surface of revolution 
of the form ^ in Nil(3) satisfies the first order differential equation 

(6) /(4 + / 2 / t 2 + 4/2)- 1/2 = C (a constant), 
and the second order differential equation 

(7) f^ + f)ftt = 4(1 + if), 

where ft and fu denote respectively the first and second derivatives of the 
function f with respect to the variable t. 

Proof. According to [5] Theorem 3, we already know that minimal surfaces 
of revolution do exist in Nil(3). Equation ((U]) is established by applying 
Proposition 12.11 with the Killing field K, = Z . The constant C can then 
be interpreted in terms of a flux. The vectors J~t and J-q are tangent to 
the surface. Using ([2]), they can be expressed in the orthonormal frame 
{X, Y, Z} at T(t, 9) as 



(8) 



f ^ = f t cos9X + f t sm9Y + Z, 
\ Tq = -fsm9X + fcos9Y-y 2 Z. 
The Riemannian measure o~c t is given by 



da Ct = ^g~{Te~Te)d9 = f\jl + \f 2 M. 

Up to sign, the vector v is characterized by the facts that it is unitary, 
tangent to the surface - hence a linear combination of Tt and Tq - and 
orthogonal to J-g. Consider the vector n = Tt + olTq with a such that 
g(n,Tg) = 0. Choose v = g(n, n)^ 1 ^ 2 n. The expression g(Z,u) which 
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appears in (0) when we choose fC = Z, is the Z-component of v. A straight- 
forward computation gives that a = 2(4 + Z 2 )" 1 , g(n, Z) = 4(4 + / 2 ) _1 and 
g{n, n) = f? + 4(4 + f 2 )' 1 . It follows that 

Using ([S]),we obtain that the quantity (a flux) 

/(t)[4 + / 2 (t)/ 2 (t) + 4/ 2 (t)]~ 1/2 

is independent of t. Equation ^ follows. Taking the derivative of © arid 
using the fact that ft ^ (see [B]), we obtain Equation ([7]). □ 

Remark. The above equations can also be derived directly from [H] (using 
the computations in the proof of their Theorem 3) or by minimizing the area 
of a rotational domain, in the spirit of the calculus of variations. 

2.3. Qualitative analysis of Equation (|7|). Given a > 0, consider the 
Cauchy problem, 

r f(f 2 + ±)fu = 4(1 + /2), 

(9) /(0) = a, 

I ft(0) = 0, 

where the subscript t means that we take the derivative with respect to 
t. Recall that this differential equation admits a first integral and, more 
precisely, that 

do) V + W + Jf) = *+j. 

f 2 a 2 

A simple analysis shows that Q admits a maximal solution f(a,t) which is 
an even function of t on some interval (— A a , A a ). Furthermore, the function 

/(a,-) : [0,4.) -> [«,°°) 
is an increasing function and we can introduce its inverse function 

4>(a, •) : [a, ex) ->■ [0,4). 
Using (|10|) . we infer that <fi is given by the integral 



a 



r/a / a 2 w 2 + 4 



(ii) *M = 2h i^T dv - 

It follows that 

(f>(a,r) ~ — r, when r — > oo. 

Finally, we conclude that the Cauchy problem ([SJ admits a global solution 
/(a, •) : ]R — > [a, oo) which satisfies 

f(a,t) = f(a,-t), 

/(M) ~ and 
ft(a,t) ~ |sgn(t), when |t| ->• oo. 



6 



P. BERARD, M. CAVALCANTE 



2.4. The Jacobi operator of minimal surfaces. In this section, we recall 
some classical definitions and facts about the Jacobi operator of minimal 
surfaces. Let M 2 S-> M 3 be an orientable minimal surface immersed into an 
oriented Riemannian manifold (M, g). Let Nm be a unit normal field along 
M, Am the second fundamental form of the immersion with respect to the 
normal Nm, and let Ric be the Ricci curvature of M. The second variation 
of the volume functional gives rise to the Jacobi operator Jm of M (see |lUj ) 

(12) J M ■■= -A M - (\A M \ 2 + mc(N M )), 

where Am is the non-positive Laplacian on M for the induced metric. 

Given a relatively compact regular domain £1 on the surface M, we let Ind(f2) 
denote the number of negative eigenvalues of Jm for the Dirichlet problem 
in S7. The Morse index of M is defined to be the supremum 

Ind(M) := sup{lnd(0); fi <s M} < oo, 

taken over all relatively compact regular domains. Let Ai(fi) be the least 
eigenvalue of the operator Jm with the Dirichlet boundary conditions in 
fi. We call a relatively compact regular domain f2 stable if Ai (SI) > 0, 
unstable if Ai(O) < 0, and stable-unstable if Ai(f2) = 0. More generally, we 
say that a domain f2 (not necessarily relatively compact) is r-stable if any 
relatively compact subdomain is stable. In the following proposition, we 
collect classical results which will be used later on. 

Proposition 2.3. Given a minimal immersion M 2 S-> M 3 , the following 
properties hold. 

(i) Let f2 be a stable-unstable relatively compact domain. Then, any 
smaller domain is stable while any larger domain is unstable. 

(ii) We refer to the solutions of the equation Jm( u ) = as Jacobi func- 
tions on M. Let X a : M 2 S-> (M 3 ,g) be a one-parameter family of 
oriented minimal immersions, with variation field V a = A^ 2 - and with 
unit normal N a . Then, the function g{V a ,N a ) is a Jacobi function 
on M. 

(iii) Let Q be a relatively compact domain on a minimal submanifold M. 
If there exists a positive function u on ft such that Jm(u) > 0, then 
Q is stable or stable-unstable. 

Proof. Assertion (i) follows from the min-max characterization of eigen- 
values and the maximum principle. Assertion (ii) appears in pQ (Theorem 
2.7 and its proof) in a more general framework. For Assertion (iii), see the 
proof of Theorem 1 in [B] . □ 

3. Stable domains of revolution on the catenoids 
We consider a catenoid C given by the map, 

T : K x [0,2vr] -> C ^ Nil(3), 



F(t,6) = (f(t)cose,f(t)amd,t), 
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where / is a global solution of ©. It follows from ([SD that the first funda- 
mental form induced by T is given by 



9T 



+ ~\f 
,~\f f 2 (l + \f 2 ) 



For later purposes, we introduce the functions 

(13) G = / 2 (1 + \f 2 ) and D = ^Det(g^) = /(l + jf + \f ft) 1 ' 2 '. 

Let N be a unit normal field to J- . Writing N = aX + j3Y + 7Z, we can 
choose N to be 



(14) 



a = W{— cos 6 - \f ft sin 6 
/? = W(-sin9 + lff t cos6 
7 = W^/tj where 



3.1. Jacobi functions coming from ambient Killing fields. Since the 
set {£, ry, (, p} is a basis of Killing vector fields, it follows from Proposi- 
tion E3Iii) that the functions 

f ^ = §(t,N) = W(-co S e + ±ff t sme), 

(15) J v v = g(r,,N) = W(-sm9 - ft cob 9), 
{ v c = g((,N) = Wf t , 

are Jacobi functions on the surface J- (note that v p = g(p, N) = 0). 

Remark. The Jacobi functions Vf,v„ and are linearly independent. 

3.2. A Jacobi function coming from the variation of the family. We 

now consider the one-parameter family of catenoids {C a ,a > 0}, associated 
with the family of maps 

(16) F{a,t,0) = (f(a,t) cos6, f {a, t) sin6, t), a > 0, 

where f(a, ■) is the unique global solution of the Cauchy problem @. The 
variational field of this family is given by 

(17) F a (a,t,9) = f a (a,t)cos9X + f a (a,t)sm6Y, 

where f a (a,t) := j^(a,t). By Proposition 12. 3l fii). this yields another Jacobi 
function on C a , namely, e(a, •) = —g{F a -,N). More precisely, 

(18) e(a,t) = (Wf a )(a,t), 

where the function W is given by the last line in (|14p . We note that e(a, •) 
does not depend on 9 and is an even function of t. Furthermore, since 
/(a, 0) = a and ft{a, 0) = 0, Va > 0, we have e(a, 0) = 1, Va > 0. 

The rotationally invariant stable domains of the catenoids C a are described 
in the following theorem. 

Theorem 3.1. Let C a be a catenoid in Nil(3). Then 

(i) The upper (resp. the lower) half catenoid C aj+ = C a n {z > 0} (resp. 
C a - = C a fl {z < 0} ) is r-stable. 
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(ii) The function e(a, •) is even and has exactly one zero z(a) on (0, oo). 
The domain J- {a, [— z(a), z(a)], [0, 2tt]) is a stable-unstable domain in 

C a - 

(iii) Given any t\ > 0, there exists some t2 > such that the domain 
T> a (—t\,t2) = -^(a, [— ti, £2]) [0, 2ir\) is stable-unstable. This implies 
in particular that both C a> + and C a - are maximal r-stable rotationally 
invariant domains (i.e. in some sense, stable-unstable). 

Proof. Assertion (i). It follows from Section [2.31 that the Jacobi function 
vq is positive on (0,+oo) and negative on (— 00, 0). The assertion follows 
from Proposition 12. 3l f hi). 

Assertion (ii). We already know that e(a, •) is an even function of t and 
that e(a, 0) = 1 for all a > 0. Claim 1. The function e(a, •) has at most 
one zero in (0,+oo). If not, e(a, •) would have two consecutive positive 
zeroes, < z\{a) < 22(a) and the domain J-"(a, [zi (a), 22(a)], [0,27r]) would 
be stable-unstable. According to Proposition I2.3f i). this would contradict 
the r-stability of C fl] + in Assertion (i). Claim 2. The function e(a, •) has at 
least one zero in (0, +00). Indeed, e(a, •) has the sign of f a (a,t). Using the 
function <f> defined by (fTT|) . we have 

0(a, /(«>*)) = t and (/>a{a,f(a,t)) + f a (a,t) 4> T (a,f(a,t)) = 

for all a,t > 0. Since 4> T is positive, it suffices to look at the sign of (ft a . We 
find that 



(19) Ma,r) 



r/a a 2 v 2 + 2 



1 x /(a 2 v 2 + A)(v 



2 




and we easily conclude that cp a (a,T) is positive when r is large enough. It 
follows that e(a,t) is negative for t large enough so that it must vanish at 
least once in (0,+oo). 

Assertion (iii). Fix some t\ > and consider the function 

w(a,t±,t) = v(a,ti)e(a,t) + e(a, ti)v(a, t), 

where we have written v(a,t) instead of v^(a,t) for short. This is a Jacobi 
function on C a , which vanishes at t = —t\. Note that w(a, t±, 0) = v (a, t\) > 
because e(a, 0) = 1 and v (a, t) > for any t > 0. As in the proof 
of Assertion (ii), Claim 1, we see that w(a,ti,-) can vanish at most once 
in (— co, 0) and (0,oo). It follows that w(a,t\,-) has exactly one zero in 
(—00, 0) - namely —t\ - and that it vanishes in (0, 00) if and only if it takes 
some negative value near infinity. Recall that 

(20) v(a,t) = ft =(a,t). 

; i + /t 2 + \Pf! 



As in the proof of Assertion (ii), Claim 2, we use the functional equations 
4>(a, f(a,t)) = t and 4> T (a, f(a,t)) ft(a,t) = 1 for all t > 0. Plugging these 
relations into (1201). we find that 



v(a,t) = v(a, f(a,t)) , Vt > 0, 
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where v(a,r) = (1 + ^ + 0^(a,r)) 1 ^ 2 . Similar computations yield the 
relation 

e(a, t) = e(a, /(a, t)) , Vi > 0, 
where e(a,r) = —cj) a (a,T)v(a,T). Define t\ := f(a,t\) and 

u)(a,ri,r) = v(a,Ti)e(a, r) + e(a,ri)u(a,r), 
so that w(a, ti, t) = w(a,Ti, f(a,t)) . Then, 

w(a,ri,r) = -u(a,r)'u(a ) Ti)(0 a (a,r) + a (a,ri)). 

Using (|19j) . we conclude that u> is negative when r approches infinity, for 
any given a, t\ > 0. This proves the existence of a positive t2 such that the 
domain T> a {— tijtz) ist stable-unstable. The last assertion follows immedi- 
ately. □ 

Remarks, (i) Consider the family of curves r a : t i— > (f(a,t),t). This 
family admits an envelope £ and the values ±z(a) correspond to the points 
at which the curve T a is tangent to £. (ii) Using (fH| and Section 12.31 we 
can see that the Gauss map of the catenoid C a covers a closed symmetric 
strip about the equator of the unit sphere in the Lie algebra £(Nil(3)). This 
strip, whose width depends on o, is strictly contained in the sphere minus 
the south and north poles. Each point of the open strip is covered exactly 
twice, except the points of the equator which are covered once (look at the 
variations of the Z-component 7 of the vector N). 

4. The index of the catenoids C a in Nil(3) 

In this section, we study the Morse index of the catenoids C a . It turns out 
that the representation T given by (UD, with the function / satisfying 0, 
is not well-adapted to Fourier analysis on C a because the vectors Ft and 
Tq are not orthogonal. To avoid this problem, we introduce a perturbed 
representation, 



T{t, 6) := Tit, 9 + <p(t)) = [f(t) cos(# + <p(t)), f{t) sin(# + <p(t)),t 
The tangent vectors are given by 

f T t (t,0) = Tt(t,6 + <p(t)) + <pt(t)Fo(t,6 + <p(t)), 
I Te(t,0) = T e (t,0 + <p(t)). 

It follows that the representation T is orthogonal - i.e. the vectors Tt and 
Tg are orthogonal - if and only if the function ip satisfies the differential 
equation 

( 21 ) <ft = iTF 

From now on, we choose ip to be the solution of (|21|) such that <p(0) = 0. 

Note that in the above expressions, we have omitted the dependence on the 
parameter a. The unit normal vector to C a at the point T(t, 6) is N(t, 9) = 
Nit, 9 + <p(t)). In the representation T, the Riemannian metric induced 
by the immersion C a Nil(3) is of the form D 2 G~ 1 dt 2 + Gd9 2 , with the 
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functions D, G as in It follows that the Laplacian on C a is given, in 

the representation F, by the expression 



*-5*(§*) + s* 



We introduce the operator 



D \D 



and the function 

V = (mc(N) + \A\ 2 ), 

which only depend on the variable t (and the parameter a) . In the parametriza- 
tion the Jacobi operator (|12|) of the immersion C a 7^ Nil(3) is given by 
the expression 

J = L-V-±d 2 ee . 
We have the following lemma. 

Lemma 4.1. With the above notations, the function V on the catenoid C a 
is given by, 

~ = 2a^ + 2(a 2 + 4) 



f 4 ( 4 + /2)2- 

i+f _l a 2 +4 _ 



Furthermore, the function GV is equal to ^ —j£ — h a2 j~ 4 jjt? satisfies 
the inequalities 



(a 2 + 2)^1- (fl2 + 2 ) 2 =a^a 2 + 4< (GV)(o,t) < a 2 + 2, 
/or all a > and all tgR. 

Proof. For the catenoid C a , the function / satisfies the differential equations 
(|1U|) and ([7]) and we have W = j, where the function W is defined in (|14|) , 

The Z-component 7 of the unit normal N is a Jacobi function which only 
depends on t, hence £(7) = Vj. Using (fTUj) and ([7]) again, we can compute 
L(7) and derive the formulas for V on the catenoid C a . The second assertion 
follows easily. □ 

Let and be the expressions of the Jacobi functions associated with the 
Killing fields £ and n in the parametrization J 7 . It follows from (|15|) that 

6) = g(af(t, 0)), N(t, 9))=W{- cos(# + ^)+ l -ff t sin(fl + ^ 

and similarly for (we have omitted the dependence on a). We introduce 
the smooth function ip(a, t) such that 

= (1 + i/ 2 / t 2 r 1/2 , 

= + |/ 2 /ir 1/2 , 
= 0. 




ROTATIONAL CATENOIDS IN THE HEISENBERG GROUPS 



11 



It follows immediately that 

v^(a,t,9) = — W±(a, t) cos (9 + ip(a, t) + ip(a, t)), 
v v (a,t,9) = — W± (a, t) sin (9 + ip(a,t) +tp(a, t)), where 



aj jti - j v 4+72 
With the above notations, we have the following lemma. 

Lemma 4.2. Let oj := (p + ip } a function of the variable t and the parameter 
a. Then, 



(22) 



(i) The functions 

wi(a,t,9) 
W2{a,t,9) 
w 3 (a,t,9) 
. W4(a,t,9) 



W\(a, t) cos(u;(a, t)) cos 9, 
W\(a, t) cos(a;(a, t)) sin 9, 
W\(a, t) sm(u(a, t)) cos 9, 
W\(a, t) sin(w(a, t)) sin 9, 



are bounded Jacobi functions on C a , J(wi) = 0, for 1 < i < 4. 

(ii) The function u>(a,-) is an odd function oft, satisfying cj(a,0) = 
and uh = 4/ 2 (/ 4 + 4a 2 )~ 1 . 

(iii) Let 0(a) := lim^ +00 u>(a, t). Then 



{1(a) = 2a [ — du. 

K ' Ja (u 4 + 4a 2 ) v / ^ 2 -a 2 

(iv) For all a > 0, we have ^ < 0(a) < tt and the lower and upper bounds 
are achieved as limits when a tends respectively to zero and infinity. 

Proof. Assertion (i) follows from the equalities = —w\ + W4, v„ = —W2 — 
u>3, and the fact that the operator J separates variables. Assertion (ii). The 
computation of uit is straightforward. To prove Assertion (iii), we use the 
fact that ft is positive for positive t and can be computed from (|l(jp . namely, 



We write 



2a/ 2 /F+4 
Ut (/ 4 + 4a 2 ) x /7 2 ^^ / * 

for t > 0, and we compute the integral f uj t dr by making the change of 
variables u = fit). Assertion (iv). Assume by contradiction that O(ao) > tt 
for some ao- There would then exist a value to such that co(ao, to) = tt. The 
function w 3 in (f22|) would then vanish on the circles .F(ao,{0}, [0, 2ir]) and 
J-(ao,{to},[0,2ir]). Because this function is a Jacobi function, this would 
contradict Assertion (i) in Theorem 13. 1[ The fact that \ < 0(a) follows 
from a direct estimate of the integral, [TT]. Indeed, making the change of 
variables u = av, we get 0(a) = Oi(4/a 2 ) where 



„ ... f°° v 2 Vv 2 + b , f°° Ju Vu + b , 

nUb) = 2 — d«= / v — r dit>J(6), 

where 

7i u z + 
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and we claim that 1(b) > %. To prove this last assertion, we consider two 
cases, < b < 1 and b > 1. 



We have 1(0) = 2 and for < b < 1, 



re 



du > 2 / — dv 



u 2 + 1 ~ Ji v 2 + l~~ 2 
When 6 > 1, we can write 



J (6) = / ~5 — T du + / -s — r du 

w Ji u 2 + b J b v? + b 

and estimate the integrals on the right-hand side separately. 

^dn > r^ 7 dn= 2 rid, 



u 2 + b J b u 2 + b JVb v* + b 

f°° 1 

> 2 / — d-y = 7T — 2 arctan vb. 

~ Jjb v 2 + 1 



On the other hand, 



du > v o / — s — r du 



i 



u 2 + b Ji u 2 + b J J- v 2 + 1 

Vb 



= arctan(\/fr) — arctan(-^=) 
= 2 arctan (Vb) — 5. 

It follows that 7(6) > ^ and hence that 0(a) > \. Recall that U(a) < it 
for geometric reasons. Clearly, when b tends to zero, tends to ir, 

and hence £l(a) tends to n when a tends to infinity. Making the change 
of variable u = Vbv, one can show that 1(6) tends to ^ when b tends to 
infinity. On the other hand, it is easy to see that £li(b) — 1(b) tends to 
when b tends to infinity. It follows that £l(a) tends to ^ when a tends to 
zero. This finishes the proof of the lemma. □ 

Lemma 4.3. For k G N ; consider the operator '■= L + jt — V in 
L 2 ([— r, r], D dt), with Dirichlet boundary conditions. Then, 

(i) For any r > 0, the operator has at most one negative eigenvalue 
(with multiplicity one). 

(ii) For all k > V a 2 + 2 and r > 0, the operator is positive. 

Proof. Assertion (i). Recall that the eigenvalues of a Sturm-Liouville prob- 
lem with Dirichlet boundary conditions are always simple. If had at least 
two negative eigenvalues, we would have an eigenfunction v of associated 
with a negative eigenvalue and having one zero in (— r, r). The function 
v cos(kO) would be an eigenfunction of the Jacobi operator J with negative 
eigenvalue, vanishing on the boundary of an annulus contained in or 
in C a -. This would contradict Assertion (i) in Theorem 13. 1[ Assertion (ii). 
By Lemma 14. 1| GV < a 2 + 2 and the second assertion follows from the 
positivity of the operator L in L 2 ([— r, r],D dt). □ 
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Theorem 4.4. Consider the catenoids {C a ,a > 0} in Nil(3). 

(i) For all a > 0, the catenoid C a has finite Morse index at least equal 
to 3 and at most equal to l + 2[\/« 2 + 2], where [x] is the integer part 
of x. In particular, the index ofC a is equal to 3 for a close to zero. 

(ii) When a tends to infinity, the index of C a grows at least like y/3a. In 
particular, it tends to infinity when a tends to infinity. 

Proof. Fourier analysis and Lemma I4.3f i) show that the Morse index of C a 
is equal to 1 plus twice the number of positive integers k such that the oper- 
ator Lfc has a negative eigenvalue. Assertion (i). The fact that the index of 
C a is at most l + 2[v / a 2 + 2] follows from Lemma l4.3f n). By Lemma [4.2l fiv). 
for any a > 0, 0(a) > 7r/2. Since w(0) = 0, there exists some r a > such 
that uj(a, r a ) = ^. The functions W\, W2 of Lemma 14.21 (i) are Jacobi func- 
tions which vanish on the boundary of the domain J- (a, (— r a , r a ), [0, 2ir]). It 
follows easily that the index of the operator L\ is equal to 1 and hence the 
index of the catenoid C a is at least 3. Assertion (ii). To determine whether 
the index of is 1 or 0, consider the associated quadratic form on functions 

/oo r 1 ~ D 

Jp<l> 2 t + (k 2 -GV)-<f> 2 }dt. 



Write (p(t) = ip(s(t)), with 

St = § = ^Twr s(0) = a 

The function s is a diffeomorphism from R onto ( — S(a), S(a)) , where 

_ f°° ft dt _ 2 f°° du 

{a) ~ 2 Jo y/{t + n<f'-a*)-*h ^( U 2 + ^ )(U 2_ 1) - 

It follows that 

S{a) 



Qk(<P)= / {rt + (k 2 -u(s))ip 2 }ds, 

where the function U is defined by U(s(t)) = (GV)(t). Choose the function 
ip to be ipo(s) = cos ( 25(q) ) an d let be the corresponding function. Using 

Lemma r4.ll one finds that Qk(<t>o) < 0, i.e. that the index of Lk is 1, as 
soon as 



(24) k 2 <(a 2 + 2)Jl 




By (f2"3"j) . S(a) = f — ^J(a), where the function J (a) is given by 

dv 



f c 

J(a) = 2 J 



v(v + Jv 2 + £)J(v 2 -l)(v 2 + £) 



This function tends to j when a tends to infinity and hence the right-hand 

side of (|24|) is equivalent to when a tends to infinity. This proves the 
second assertion. □ 



11 



P. BERARD, M. CAVALCANTE 



Remarks. 

(i) Given a > 0, there is a simple criterion to decide whether the opera- 
tor Lfc has a negative eigenvalue in the interval [— r, r] (with Dirichlet 
boundary conditions). Let be the solution of the Cauchy problem 
Lk{u) = 0, u(0) = 1 and ut(0) = 0. If u& has a zero in the interval 
(0, r), then Lf, has a negative eigenvalue in [— r, r]; if uj~ does not 
vanish in the interval (0, r), then L^iu) > in [— r, r\. 

(ii) Using the fact that the metric g on Nil(3) is left-invariant, one can 
easily express the associated Levi-Civita connexion and curvature 
tensors in the orthonormal basis {X, Y, Z} of left-invariant vector 
fields. In particular, given a unit vector N = aX + (3Y + 7Z, we 
find the following formula for the Ricci curvature, 

mc(iv,iv) = -i + 7 2 . 

(iii) Using the preceding remark, we can write the Jacobi operator on an 
orientable minimal surface in Nil(3) as 

J = -^ + \-l 2 -\A\\ 

where 7 is the Z-component of the unit normal to the surface. Using 
the fact that the scalar curvature of Nil(3) is — |, we also have the 
formula 

J = -^ + \ + K M - l -\A\\ 

where Km is the Gauss curvature of the surface M. 

(iv) Using Lemma 14.11 and the second remark, we deduce the following 
expression for the second fundamental form of the catenoid C a in 
Nil(3), 

U|2 = i_± 4(a 2 + 4) 2(a 2 + 4) 

I I 2 /2 + /2(/ 2 + 4) + (/ 2 +4) 2- 

This shows that the norm squared of the second fundamental form 
tends to ^ uniformly at infinity. This is in contrast with the situation 
in R 3 ,M 2 x R or M 3 . 

5. Catenoids in higher dimensions 

In this section, we study the rotationally symmetric stable domains on the 
higher dimensional catenoids. Let Nil(2n + 1) be the (2n + l)-dimensional 
Heisenberg group. As in Section [21 we use the exponential coordinates and 
choose the left-invariant metric g to be such that the left-invariant vector 
fields {Xi, • • • , X n , Y\, ■ ■ ■ , Y n , Z} form an orthonormal basis, where 

Xi(x,y,z) = d Xi - \yid z , 1 < i < n, 

Yi(x,y,z) = d Vi + \xid z , 1 < i < n, 

Z(x,y,z) = d z . 

We look for hypersurfaces of revolution of the form 

[Rx S 2 ™" 1 -> Nil(2n + 1), 
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where / is a positive function of t. If follows from 0[B] that the hypersurface 
T is minimal if and only if / satisfies the second order differential equation, 

7(4 + f)ftt = 4(2 ? i - 1)(1 + ft) + (2n - 2)/ 2 / 2 . 

As in Section 12. 3\ one can show that for a > 0, there is a unique maximal 
solution f(a,t) such that /(a, 0) = a and ft(a,0) = 0. This is an even 
function of t defined on the interval (— T(a), T(a)), where T(a) is finite 
when n > 2. As in dimension 3 (n = 1), the above differential equation 
admits a first integral, 

/2 n-l (l + 7 2 + /2/ 2 r V 2 ^ Q 2n-1 

As in USD, we let W := (1 + / t 2 + / 2 / 2 )~ 1/2 . We also use the following 
notations, 

c a = ^.(-rca),^)),^ 1 ), 

Ca,+ = ^(a,(0,T(a)),5 2n - 1 ), 
Ca,- = J-(a,(-r(a),0),S 2n - 1 ), 
t 2? a (r,a) = ^(^(r^)^ 2 "- 1 ). 
We can now state the following result. 

Theorem 5.1. Assume that n > 2 and a > 0. 

(i) The half-catenoids C a) ± are r- stable. 

(ii) There exists some z{a) > such that the domain V a (—z(a),z(a)) is 
stable-unstable. In particular, the catenoid C a has index at least 1. 

(iii) There exists some 1(a) > such that the domain D a (—£(a),T(a)) is 
r-stable. 

(iv) For any r > £(a), there exists some s > such that the domain 
V a (—r,s) is stable-unstable. 

Proof. The proof relies on the expressions of two explicit Jacobi functions on 
C a , namely the Jacobi functions v(a,t) = g(N, Z), and e(a,t) = — g(T a , N), 
where N is a unit normal to C a , and J- a is the variation field along T when 
the parameter a varies. As in dimension 2, we have v(a,t) = W(a,t)ft(a,t) 
and Assertion (i) follows immediately from the fact that ft(a,t) > for 
t > 0. 

To prove the other Assertions, notice that e(a, t) is an even function of 
t which can be studied using the inverse function (j>(a, r) of the function 
/(a,-) : [0, oo) — > [a, T(a)). It turns out that 

u 2 + 4 

An-2 _ a 4n-2 

This formula shows that (j>(a, t) has a finite limit T(a) when r tends to 
infinity and that its derivative (p a (a, t) has a positive finite limit when r tends 
to infinity. We now use the same method as in the proof of Theorem 13.11 
Assertion (ii), follows from the fact that e(a,0) = 1 and that e(a,t) takes 
negative values near infinity. For the proofs of Assertions (iii) and (iv), we 
use the fact that in higher dimensions (n > 2), both <j)(a,r) and <p a (a,r) 
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have finite limits at infinity, so that the higher dimensional case differs from 
the case in which n = 1. □ 

Remark. Theorem 13.1 ( in) tells us that the half-catenoids C tt) ± in Nil(3) are 
stable-unstable, i.e. that they satisfy the Lindeloef's property as defined in 
[21 E]. Theorem I5.1f iii) and (iv) tell us that catenoids in Nil(2n+ 1), n > 2, 
do not satisfy Lindeloef's property. As for catenoids in R n+2 and IH n x R, 
n > 2, this is related to the fact that these catenoids have finite height. 
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